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Abstract: Faculty and assessment professionals rely on Bloom's taxonomy to guide them when 
they write measurable student learning outcomes and describe their goals for developing students' 
thinking skills. Over the past ten years, assessment offices and teaching and learning centers have 
compiled lists of measurable verbs aligned with the six categories that comprise Bloom's taxonomy. 
The author analyzed 30 compilations posted on web sites and evaluated how well these verbs 
aligned with categories in Bloom's taxonomy. The author discusses the value of Bloom's taxonomy 
as a heuristic for writing student learning outcomes and other factors faculty should consider when 
they articulate learning outcomes to describe levels of expertise attained by students who complete 
an associate's, bachelor's, or graduate degree. 
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1. Introduction 

Student learning outcomes (SLOs) serve several purposes. The most explicit goal is to describe 
the unique knowledge, skills, and abilities students acquire as they complete courses and degree 
programs. Two less explicit goals are to support the design of curriculum and align learning goals with 
learning activities and assessments [1,2]. An additional goal, which is an important component of the 
Degree Qualifications Profile, is the articulation of how students who earn advanced degrees differ in 
professional expertise and skill from students who earn less advanced credentials [3]. Faculty appeal 
to Bloom's taxonomy [4] and models of cognitive development in higher education (e.g.. Perry [5] 
and Clinchy [6]) to achieve these goals. This paper examines how well current recommendations for 
writing SLOs based on Bloom's taxonomy serve these goals. 

Bloom's taxonomy classifies thinking skills into six hierarchically organized categories that range 
from lower-level cognitive skills (know and understand) through higher-order cognitive skills (apply, 
analyze, evaluate, create) based on the verbs teachers select when they describe expectations for 
students' thinking skills and behaviors in a learning outcome [4]. Since its publication, teachers have 
relied on Bloom's taxonomy to guide how they write learning outcomes, structure learning activities, 
and assess student learning. Bloom's taxonomy guides the development of test questions to assess 
higher-level thinking skills by drawing attention to what test questions and assessment prompts 
require students to do (retrieve facts, apply knowledge, make a prediction, solve a problem, or evaluate 
a theory). Thus, Bloom's taxonomy has influenced how instructors design their courses, how they 
describe learning outcomes, and how they create assessments of learning. 

Ideally, SLOs use language that can be understood by colleagues, students, parents, and potential 
employers. SLOs written with verbs that describe specific, concrete actions or operations minimize 
ambiguity. They describe what the student can do in terms of specific, observable behaviors and 
communicate learning goals in clear, jargon-free language [7]. 
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Student learning outcomes are analogous to operational definitions in the natural and 
social sciences. The criteria that characterize a measurable SLO are similar to those that 
characterize the operational definitions researchers create to design experiments and address research 
questions [8]. Operational definitions describe the procedures researchers create when they manipulate 
an independent variable and describe the instruments, operations, procedures, and public behaviors 
they use to measure dependent variables. SLOs that describe learning in operational (measurable) 
language create a link between faculty intentions for student learning and the strategies faculty will 
use to assess learning. Student learning outcomes written with objective action verbs imply the kinds 
of assignments students complete and the kinds of assessment measures these assignments generate. 
The assessments will be grounded in student behaviors that instructors can observe directly or in 
a tangible product students create and instructors evaluate on specific dimensions of quality [8]. 

Guidelines built around Bloom's taxonomy assume the verbs in each category describe 
a progressive development of cognitive skills. Verbs at lower levels of Bloom's taxonomy 
describe acquisition of knowledge and facts whereas verbs at higher levels of Bloom's taxonomy 
describe complex thinking skills, including application of knowledge to practical problems, analysis 
of competing interpretations, and creation of new knowledge or alternative interpretations of 
existing findings. 

Other taxonomies focus on the relation between learning outcomes, learning activities, and 
assessments without making claims about the nature of the thinking skills described [1,7]. For example, 
Fink proposes a taxonomy of six categories he believes are necessary to create "significant learning" [1]. 
Fink makes no claims about how the verbs from categories in his taxonomy relate to the development 
of cognitive skill. Fink emphasizes the importance of aligning learning goals with learning activities 
and assessments, using a process called backward design [2]. An instructor who uses backward 
design to create a course will identify the course learning outcomes first. The instructor then selects 
reading materials and designs learning activities and assignments to create opportunities for students 
to acquire and practice these skills. Finally, the instructor selects assessment instruments that evaluate 
the learning outcomes. In contrast, an instructor who does not use backward design might begin 
designing a course by selecting assigned readings and determine afterward what students might learn 
from those texts. Like Fink, Adelman proposes a taxonomy comprised of nearly 20 categories of verbs 
based on functional activities aligned with specific cognitive demands of assignments and projects [7], 
Instead of organizing verbs into categories of thinking skill, Adelman's approach focuses on creating 
categories of verbs that enable faculty to align course and program goals with instructional strategies, 
assignments, and assessments, as occurs when instructor use backward design to create courses [1,2]. 

Differentiating between types of degrees within a discipline is a more challenging task. Faculty 
often define higher order thinking skills that describe learning in terms of Bloom's taxonomy when 
they write SLOs for advanced academic work. They justify this practice by arguing that the categories 
represent a hierarchy of increasingly complex skills [4] . However, Anderson and Krathwohl raised 
questions about the developmental sequence of categories in Bloom's hierarchy [9]. Paul also criticized 
the hierarchical structure of Bloom's taxonomy, arguing that critical thinking skills permeate every 
category of Bloom's taxonomy [10]. Fink and Adelman make no assumptions about how student 
learning progresses across their categories [1,7]. Adelman argues that contextual information such as 
how well a student executes skilled behavior and the disciplinary content itself may be necessary to 
differentiate between learning outcomes for associate's, bachelor's and master's degrees. 

The SLOs for different degree programs should illustrate how the skills of a graduate in 
one program or discipline differ from those of a graduate in another program. SLOs should also 
describe the knowledge, skills, and abilities that characterize the expertise represented by the award 
of specific degrees within a discipline (associate's, bachelor's, or graduate degrees). In addition, 
accrediting bodies establish standards that require institutions to differentiate the learning goals they 
articulate for associate's, bachelor's, graduate, and other degree credentials. The SLOs that describe 
the skills attained by a student who completes a more advanced degree should reflect a different level 
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of expertise than the SLOs written for a less advanced degree. The Degree Qualifications Profile (DQP) 
aspires to describe how students who merit the award of an associate's, bachelor's, or master's degree 
differ in the kinds of skills students achieve [3]. The DQP is offered as a model an institution might 
use to frame program learning outcomes in specific fields and articulate the learning expectations 
associated with each type of degree program, independent of the discipline in which the degree is 
awarded [3,11]. 

The internet includes many resources that offer guidelines on how to write measurable SLOs. 
Many of these resources provide examples of measurable verbs, usually based on Bloom's taxonomy. 
I created a collection of verbs for Bloom's taxonomy by aggregating words gathered from over a dozen 
web sites, each of which provided examples of verbs for each level of Bloom's taxonomy. In 2006, 
when I created my collection, few web resources posted large lists of words. Most sites identified 
8-15 words for each level of Bloom's taxonomy. Large collections of verbs for Bloom's taxonomy have 
proliferated in recent years. A Google search will now produce many examples of what Adelman 
describes as "formless lumps of verbs" for Bloom's taxonomy [7]. Although some sites mirror lists 
created by others, many sites provide their own collection. 

I was curious about whether these collections included verbs I might have overlooked when 
I created a collection of verbs for Bloom's taxonomy. The sources I consulted sometimes placed 
a particular verb in different taxonomy categories. As a result, a few verbs in my collection appeared 
in more than one category because different sources aligned these verbs with different categories. 
Did the authors of other large collections also encounter this duplication of verbs across categories? 
If so, how frequently did lists align verbs with the same specific categories? Would an analysis of these 
collections identify converging opinion about the most appropriate alignment of a verb with a specific 
category in Bloom's taxonomy? Thus, I posed the following research questions: 

• How consistently do authors use specific verbs to describe learning at different levels of 
Bloom's taxonomy? 

• Will an analysis of large collections of verbs for Bloom's taxonomy produce consensus about the 
level of thinking skill described by an SLO that uses a specific verb? 

• Can we make unambiguous recommendations about which verbs correspond to specific levels of 
cognitive skill implied by the structure of Bloom's taxonomy? 

2. Materials and Methods 

Materials were obtained by conducting a Google search based on the string, "action words for 
Bloom's taxonomy", in March 2016 and downloading the lists posted in the top 30 web sites produced 
by this search. The 30 highest-ranking web sites each contained a unique collection of words. However, 
sites with lower rankings in the Google search frequently mirrored a list posted by one of the top 
30 sites. The web sites that provided a list of Bloom's taxonomy verbs used for this analysis are 
identified in Appendix A. 

Words harvested from lists posted on the 30 highest-ranking sites in the Google search were 
aggregated in a single data set without regard to duplicated words within or across levels of Bloom's 
taxonomy. Each time a list identified a word for a given level of Bloom's taxonomy, the word was 
recorded for that category in the data set. If a list duplicated a word, identifying it in more than 
one category, the word was recorded in the data set for each category in which it was listed. 

Words that were not verbs were deleted (e.g., hozv, what, when, where, which, who, why). Grammatical 
variations for verbs were standardized. For example, words that were nominalizations were converted 
to their verb form (e.g., organization became organize, retrieving became retrieve, theory became theorize, 
inference became infer). 

Citation frequency, the number of lists that assigned a verb to a given level of Bloom's taxonomy, 
was recorded for verbs in each of Bloom's categories. Citations frequency values could range from 1 
(only one list aligned the verb with that category) to 30 (every list aligned the verb with that category). 


Educ. Sci. 2016, 6, 37 


4 of 12 


After recording the citation frequency for each verb, duplicate copies were deleted within a category 
(but a duplicate copy of the verb was retained in other categories where it occurred). 

Verbs were sorted alphabetically within categories to identify how often a verb was duplicated 
across categories. That is, how often a verb appeared in two or more categories. This value could vary 
from 1 (verbs assigned to a single category) to 6 (verbs assigned to every category). The number of 
verbs assigned to a single category and to varying numbers of categories was recorded. Percentage 
of verbs assigned to a single category was used as an estimate of consensus across lists about 
category membership. This analysis was completed with verbs organized in six categories (based on 
Bloom's taxonomy) and when verbs were organized in three categories (fact-based/low-level thinking, 
application, and conceptual/high-level thinking), which were created by merging selected categories. 

Finally, verbs within each category were sorted by citation frequency to create smaller collections 
comprised of verbs that multiple lists nominated as members of a category. The percentage of verbs 
assigned to a single category in these smaller lists evaluated the consensus among lists about category 
membership. The most lenient criterion selected verbs nominated by four or more lists. A more 
stringent criterion selected verbs nominated by 10 or more lists. These two analyses determined 
whether list consensus improved when more lists agreed that the verb belonged to a given category. 

3. Results 

The full collection, based on all verbs included in the 30 lists (788 verbs), included duplications 
within categories (created when several lists aligned a verb to the Bloom's taxonomy category) and 
duplications across categories (created when different lists associated a verb with different categories 
or when a single list associated a verb with more than one category). No verb was assigned consistently 
to the same category by all 30 lists in the sample. Many lists in the sample duplicated verbs and listed 
them in two or more categories. 

After eliminating nouns and converting nominalizations to verbs, the final data set was edited 
to eliminate within-category duplications. This operation reduced the collection to 433 unique verbs, 
of which 236 verbs (54.5%) appeared in only one category. The remaining 197 verbs (45.5%) appeared 
in two to six categories: 108 verbs (24.9%) appeared in two categories, 41 verbs (9.5%) appeared in three 
categories, 30 verbs (6.9%) appeared in four categories, 15 verbs (3.5%) appeared in five categories, 
and 3 verbs ( choose , relate, select) appeared in all six categories. Thus, analysis of the verbs included on 
30 lists indicates that list authors vary widely in how they interpret the thinking skills described by 
these verbs. 

Researchers establish 75% agreement as the threshold criterion for acceptable inter-rater agreement 
for assessment instruments [12,13]. Agreement among Bloom's taxonomy lists might be defined in 
two ways. First, inter-list agreement might be defined as the percentage of lists that agree in assigning 
a verb to a category. A second definition might be defined in terms of intra-list agreement or the 
internal consistency of a collection of verbs. In this case, agreement is defined in terms of the percentage 
of verbs in a collection assigned to only one category. 

Given the minimal level of within-list agreement (the percentage of verbs placed in only one 
category) observed for the collection based on all 30 lists, the collection was reduced to include only 
those verbs nominated for a category by at least 4 lists (representing 13.3% inter-list agreement). 
The reduced collection (176 verbs) appears in Table 1. Although less variable than the larger collection, 
this collection shows considerable variability in how specific verbs align with categories. This list 
includes 118 verbs (67%) that appear in only one category. The remaining 58 verbs appear in two to five 
different categories. Of these, 38 (21.6%) appear in two categories, 13 (7.4%) appear in three categories, 
5 (2.8%) appear in four categories, and 2 (1.1%) appear in five categories. 


Educ. Sci. 2016, 6, 37 


5 of 12 


Table 1. Sample of 176 unique words identified for a level of Bloom's taxonomy by 4 or more lists in a sample of 30 published lists (f = number of lists that nominate 
the word for a level of Bloom's taxonomy). 


Knowledge 

/ 

Understand 

/ 

Apply 

/ 

Analyze 

/ 

Evaluate 

/ 

Create 

/ 

arrange 

6 

articulate 

4 

act 

19 

analyze 

24 

appraise 

22 

arrange 

22 

choose 

4 

associate 

4 

adapt 

4 

appraise 

11 

argue 

12 

assemble 

14 

cite 

17 

characterize 

4 

apply 

22 

break 

8 

arrange 

5 

categorize 

7 

copy 

4 

cite 

4 

back/back up 

5 

break down 

7 

assess 

17 

choose 

7 

define 

21 

clarify 

5 

calculate 

10 

calculate 

9 

attach 

4 

collect 

9 

describe 

14 

classify 

18 

change 

9 

categorize 

19 

choose 

10 

combine 

14 

draw 

5 

compare 

11 

choose 

11 

classify 

10 

compare 

18 

compile 

7 

duplicate 

7 

contrast 

7 

classify 

6 

compare 

24 

conclude 

13 

compose 

19 

identify 

20 

convert 

13 

complete 

5 

conclude 

6 

contrast 

8 

construct 

29 

indicate 

4 

defend 

12 

compute 

10 

contrast 

19 

core 

6 

create 

19 

label 

21 

demonstrate 

6 

construct 

13 

correlate 

5 

counsel 

4 

design 

24 

list 

27 

describe 

22 

demonstrate 

20 

criticize 

11 

create 

4 

develop 

18 

locate 

10 

differentiate 

8 

develop 

4 

debate 

8 

criticize 

11 

devise 

13 

match 

14 

discuss 

21 

discover 

8 

deduce 

6 

critique 

14 

estimate 

5 

memorize 

10 

distinguish 

12 

dramatize 

16 

detect 

7 

decide 

4 

evaluate 

4 

name 

22 

estimate 

11 

employ 

16 

diagnose 

4 

defend 

15 

explain 

8 

order 

5 

explain 

28 

experiment 

6 

diagram 

12 

describe 

4 

facilitate 

4 

outline 

11 

express 

17 

explain 

5 

differentiate 

20 

design 

4 

formulate 

18 

quote 

7 

extend 

11 

generalize 

5 

discover 

4 

determine 

6 

generalize 

7 

read 

4 

extrapolate 

5 

identify 

4 

discriminate 

11 

discriminate 9 

generate 

11 

recall 

24 

generalize 

11 

illustrate 

18 

dissect 

6 

estimate 

15 

hypothesize 

8 

recite 

12 

give 

4 

implement 

4 

distinguish 

21 

evaluate 

16 

improve 

5 

recognize 

14 

give examples 

8 

interpret 

15 

divide 

12 

explain 

9 

integrate 

4 

record 

13 

identify 

14 

interview 

6 

evaluate 

4 

grade 

4 

invent 

10 

relate 

11 

illustrate 

9 

manipulate 

10 

examine 

18 

invent 

8 

make 

6 

repeat 

20 

indicate 

8 

modify 

12 

experiment 

9 

judge 

25 

manage 

8 

reproduce 

11 

infer 

15 

operate 

17 

figure 

4 

manage 

15 

modify 

10 

review 

4 

interpolate 

5 

organize 

4 

group 

4 

mediate 

9 

organize 

21 

select 

16 

interpret 

17 

paint 

4 

identify 

7 

prepare 

12 

originate 

9 

state 

23 

locate 

10 

practice 

15 

illustrate 

8 

probe 

4 

plan 

21 

tabulate 

4 

match 

7 

predict 

9 

infer 

14 

rate 

5 

predict 

8 

tell 

4 

observe 

5 

prepare 

11 

inspect 

8 

rearrange 

19 

prepare 

12 

underline 

7 

organize 

5 

produce 

13 

inventory 

9 

reconcile 

12 

produce 

13 
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Table 1. Cont. 


Knowledge 

/ 

Understand 

/ 

Apply 

/ 

Analyze 

/ 

Evaluate 

/ 

Create 

/ 

write 

5 

paraphrase 

22 

relate 

12 

investigate 

7 

release 

6 

propose 

9 



predict 

12 

schedule 

11 

order 

5 

rewrite 

4 

rate 

21 



recognize 

11 

select 

4 

organize 

6 

select 

5 

rearrange 

8 



relate 

7 

show 

13 

outline 

10 

set up 

15 

reconstruct 

9 



report 

10 

simulate 

5 

point out 

12 

supervise 

9 

relate 

8 



represent 

4 

sketch 

17 

predict 

4 

synthesize 

16 

reorganize 

9 



restate 

15 

solve 

19 

prioritize 

4 

test 

8 

revise 

12 



review 

15 

translate 

5 

question 

12 

value 

7 

rewrite 

7 



rewrite 

12 

use 

25 

relate 

17 

verify 

9 

role-play 

4 



select 

7 

utilize 

4 

select 

12 

weigh 

5 

set up 

9 



summarize 

20 

write 

5 

separate 

10 



specify 

5 



tell 

7 



solve 

8 



summarize 

7 



translate 

21 



subdivide 

10 



synthesize 

4 







survey 

7 



tell/tell why 

5 







test 

14 



write 

17 
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A more conservative selection criterion selected verbs that had been nominated by 10 or more lists 
for a given category in Bloom's taxonomy (representing 33.3% inter-list agreement). Table 2 presents 
this collection of 104 verbs. Although this "stricter" criterion was relatively weak in that it required 
only a third of the source lists to agree in nominating a verb for inclusion, this criterion eliminated 
nearly 76% of the verbs included in the original collection of 433 verbs. If we define consensus as 
the percentage of lists that agree on the alignment of verbs to a category, the resulting agreement 
(33.3%) is nowhere near acceptable levels established for research purposes. However, if consensus 
is defined in terms of within-list agreement, the estimate of agreement improves. This list includes 
83 verbs (79.8%) that appear in only one category. However, 21 words (20.2%) appear in more than 
one category, 18 words (17.3%) appear in two categories and 3 words (2.9%) appear in three categories. 
Thus, adopting a criterion for inclusion that required 33.3% agreement among the lists produced 
acceptable levels of agreement about the alignment of words with categories in Bloom's taxonomy but 
eliminated many serviceable and measurable verbs. The reduced collection continued to give evidence 
of multiple interpretations of the level of skill described by a verb. 


Table 2. Sample of 104 unique words identified for a level of Bloom's taxonomy by 10 or more lists 
in a sample of 30 published lists (f = number of lists that nominate the word for a level of Bloom's 
taxonomy). 


Knowledge 

/ 

Understand 

/ 

Apply 

/ 

Analyze 

/ 

Evaluate 

/ 

Create 

/ 

cite 

17 

classify 

18 

act 

19 

analyze 

24 

appraise 

22 

arrange 

22 

define 

21 

compare 

11 

apply 

22 

appraise 

11 

argue 

12 

assemble 

14 

describe 

14 

convert 

13 

calculate 

10 

categorize 

19 

assess 

17 

combine 

14 

identify 

20 

defend 

12 

choose 

11 

classify 

10 

choose 

10 

compose 

19 

label 

21 

describe 

22 

compute 

10 

compare 

24 

compare 

18 

construct 

29 

list 

27 

discuss 

21 

construct 

13 

contrast 

19 

conclude 

13 

create 

19 

locate 

10 

distinguish 

12 

demonstrate 

20 

criticize 

11 

criticize 

11 

design 

24 

match 

14 

estimate 

11 

dramatize 

16 

diagram 

12 

critique 

14 

develop 

18 

memorize 

10 

explain 

28 

employ 

16 

differentiate 

20 

defend 

15 

devise 

13 

name 

22 

express 

17 

illustrate 

18 

discriminate 

11 

estimate 

15 

formulate 

18 

outline 

11 

extend 

11 

interpret 

15 

distinguish 

21 

evaluate 

16 

generate 

11 

recall 

24 

generalize 

11 

manipulate 

10 

divide 

12 

judge 

25 

invent 

10 

recite 

12 

identify 

14 

modify 

12 

examine 

18 

manage 

15 

modify 

10 

recognize 

14 

infer 

15 

operate 

17 

infer 

14 

prepare 

12 

organize 

21 

record 

13 

interpret 

17 

practice 

15 

outline 

10 

rearrange 

19 

plan 

21 

relate 

11 

locate 

10 

prepare 

11 

point out 

12 

reconcile 

12 

prepare 

12 

repeat 

20 

paraphrase 

22 

produce 

13 

question 

12 

setup 

15 

produce 

13 

reproduce 

11 

predict 

12 

relate 

12 

relate 

17 

synthesize 

16 

rate 

21 

select 

16 

recognize 

11 

schedule 

11 

select 

12 



revise 

12 

state 

23 

report 

10 

show 

13 

separate 

10 



write 

17 



restate 

15 

sketch 

17 

subdivide 

10 







review 

15 

solve 

19 

test 

14 







rewrite 

12 

use 

25 









summarize 

20 











translate 

21 










Perhaps the six categories in Bloom's taxonomy create too many distinctions for consistent 
judgments about category membership. Would a list based on a smaller set of categories reveal more 
consistent interpretation of verb meaning? When faced with low reliability on a scoring rubric, we can 
improve reliability by asking reviewers to make fewer or simpler decisions by assigning observations 
to fewer categories. One justification for examining a taxonomy based on fewer categories is the fact 
that publisher test banks sometimes categorize multiple choice questions as fact-based, application, 
or conceptual questions, implying three levels of thinking skill. Increased consistency may come at the 
cost of making fewer discriminations. If we create a blunter instrument for describing SLOs, will we 
gain consistency? To answer this question, the data were examined after combining the categories know 
and understand to create a single category, equivalent to the concept associated with factual questions 
in a test bank. In addition, a category based on the top three categories ( analyze , evaluate, create) created 
the equivalent of the concept associated with conceptual questions in a test bank. Verbs in the application 
category were retained to represent application questions in a test bank. 
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Based on a three-category organization, 304 (70.2%) of the 433 verbs in the full collection were 
associated with a unique category, 91 (21%) were associated with two categories, and 38 (8.8%) were 
associated with all three categories. Thus, organizing the verbs in three categories produced only 
a minimal reduction in ambiguity. Moreover, the 38 verbs that aligned least consistently, appearing in 
all three categories, were verbs that frequently appeared on posted lists. In some cases, these verbs 
were nominated for category membership by 15 or more lists in the sample. Moreover, nearly all of 
these verbs were nominated by 10 or more lists to each of two categories. Clearly, authors of SLOs 
employ multiple meanings for these verbs. 

Requiring greater inter-list agreement about including a verb in a category did not improve the 
internal consistency of the resulting collection. Analysis of the 104 verbs identified by 10 or more 
lists using the three-category organization (fact-based , application, or conceptual) produced even lower 
within-list agreement: 28 verbs (26.9%) appeared in all three categories, 39 verbs (37.5%) appeared in 
two categories, and only 37 verbs (35.6%) appeared in one category. 

4. Discussion 

Adelman provides a comprehensive analysis of the challenges faculty face when they set out to 
write meaningful, measurable SLOs [7]. His analysis primarily focuses on the challenge of writing 
measurable SLOs, although he briefly discusses the role of context for articulating differences in 
disciplinary skills. SLOs for the Degree Qualifications Profile attempt to differentiate among degree 
programs by using measurable language for SLOs based on action verbs. Adelman proposes nearly 
20 sets of "operational verbs" that establish clear connections between expectations about student 
learning and strategies for assessing this learning. However, Adelman offers scant advice on how 
using these categories to write SLOs will help differentiate between learning expectations represented 
by an associate's, bachelor's, or master's level degree. 

If verbs were consistently aligned with specific levels of Bloom's taxonomy, this taxonomy would 
reduce ambiguity about levels of expertise instructors describe when they select language to write 
learning outcomes about the academic goals for a course. However, the analysis reported here suggests 
that existing lists of words organized by level of Bloom's taxonomy cannot unambiguously support 
decisions about the differences in learning expected and distinguish between academic achievement 
expected for associate's, bachelor's, and master's degree programs. 

The challenge faced by verb-based frameworks is that language is notoriously flexible. Context 
modifies meaning, which explains why a verb associated with a low level in Bloom's taxonomy in 
one context ( recognize the definition of technical terms) may be associated with a high level of Bloom's 
taxonomy in another context ( recognize professional situations that produce a conflict of interest). 
Independent of context, many words have several meanings, which contributes to ambiguity about the 
level of cognitive skill intended. For example, although we might argue that rewrite means write again, 
an author's intended meaning might correspond to a low-level skill (copy or transcribe) or a high-level 
skill ( revise or edit). Paul makes a similar argument against categorizing recall as a low-level cognitive 
skill [10]. He describes circumstances in which recall represents high-level thinking processes, arguing 
that unless the intent of recall is limited to rote recall or mechanical repetition or reproduction of content, 
recall entails complex judgments about the credibility of the content accepted as part of a student's 
belief system. Paul's interpretation of the cognitive skills embedded in one simple verb casts doubt on 
the power of verbs alone to specify the level of expertise described by an SLO. 

One solution might be to abandon verbs with multiple interpretations and select verbs with more 
precise meanings instead. However, this solution eliminates many wonderful, measurable verbs that 
serve well to articulate learning in language suitable for a broad audience. 

Degree-granting institutions must clearly articulate how learning differs among students who 
earn the various degrees awarded. Regional accrediting bodies in the United States establish 
standards that require institutions to differentiate the learning goals articulated for undergraduate, 
graduate, and other degree programs [14—19]. This differentiation includes providing evidence that 
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graduate programs "advance the student substantially beyond the educational accomplishments 
of a baccalaureate degree program" and "reflect the high level of complexity, specialization, and 
generalization inherent in advanced academic study" [16]. Learning outcomes for graduate programs 
should be "progressively more advanced in academic content" [18] and "require greater depth of 
study and increased demands on student intellectual or creative capacities" [19]. Additional language 
in accreditation standards describes "knowledge of the literature" of the discipline or field and 
"ongoing engagement" in research, scholarship, creative expression, and /or professional practice 
and training [18,19]. These standards allude to increased intellectual demands without mandating 
particular language for learning outcomes for degree programs. 

Academics argue that undergraduate students learn under the guidance of an expert, master's 
students learn to teach themselves disciplinary knowledge and skills, and doctoral students learn to 
create new disciplinary knowledge [20]. Academic programs often try to use Bloom's taxonomy as 
a way to describe the expectations for increased expertise students exhibit in advanced academic work. 
SLOs for introductory courses (lower-level or general education courses) might be described using 
verbs selected from lower levels of Bloom's taxonomy (knozv, understand, apply). SLOs for advanced 
courses describe higher order thinking skills aligned with the upper levels of Bloom's taxonomy 
(analyze, evaluate, create). Based on this framework, graduate programs should describe SLOs that 
are dominated by verbs from high levels of Bloom's taxonomy. However, this alignment is not rigid 
or formulaic. Introductory courses might include SLOs that describe higher-order thinking skills 
because these courses create opportunities for students to develop and practice skills expected at more 
advanced levels. In contrast, SLOs written for advanced courses might describe proficient use of these 
higher-order skills. In addition, advanced courses might include SLOs that use verbs from low levels 
of Bloom's taxonomy to describe the initial acquisition of advanced disciplinary knowledge and skills 
that might be considered to be too challenging to introduce to novices in the field. 

Perry and Clinchy describe models for the development of cognitive skill that suggest 
an alignment between academic programs and increasingly advanced thinking styles that could 
be described using verbs selected from levels of Bloom's taxonomy [5,6]. For example. Perry argues 
that beginning students display dualistic thinking, which treats knowledge as a cannon of known 
"right answers". Dualistic students focus on acquiring and reproducing the "correct facts". They 
rely on an expert authority (the teacher) to tell them which facts or answers are correct and suitable 
for memorization. Clinchy calls these learners "received knowers" because these students receive 
knowledge in a passive way from an authority. SLOs that describe activities such as the retention 
and reproduction of facts, drawing from Bloom's categories of knozv and understand, align with the 
cognitive skills and dispositions of dualistic learners. 

More advanced students (those in Perry's "multiplicity" stage or "subjective knowledge" in 
Clinchy's model) discover that authorities consider multiple opinions and points of view as legitimate. 
The "right" answers might be undetermined. In the multiplicity stage, students treat all opinions as 
equally valid. More advanced students, who are relativistic thinkers, realize that some opinions are 
more valued than others. Eventually, students develop agency and make rational choices about the 
relative merit of competing opinions. These students use disciplinary-specific criteria about evidence 
and argument to make decisions and build arguments in favor of one interpretation. Clinchy refers to 
this cognitive stage in terms of "procedural knowledge". SLOs that describe learning for students in 
these later stages would employ verbs associated with higher-level categories in Bloom's taxonomy 
(analyze, evaluate, create). 

Bloom's taxonomy is a useful heuristic for describing the cognitive demands imposed by learning 
tasks and assessments [4]. For example. Bloom argued that teachers who write objective exam 
questions can use the taxonomy to determine whether questions require only a superficial knowledge 
of content (recalling facts, retrieving the definitions of technical terms) or require advanced thinking 
skills (solving a problem, interpreting evidence, predicting an outcome based on a theoretical model). 
In the context of a specific question or task, the action verbs we associate with levels of Bloom's 
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taxonomy have specific and unambiguous meanings. Taken out of context, these verbs lose their 
specificity. However, in the context created by other verbs (e.g., a list of verbs for a taxonomy), a verb 
might take on one meaning, but may acquire a different meaning in the context created by a different 
set of verbs identified for a different level in the taxonomy. The context created by the object of an SLO 
also creates specificity of meaning. Thus, reviewers must consider both the verb and the remaining 
content of an SLO when they make judgments about the level of expertise an SLO describes. 

Another approach to distinguishing between academic degrees is reflected in the language of 
accreditation standards. Although accreditation standards for graduate programs describe expectations 
for increased intellectual demands in graduate level programs, they decline to specify language 
for student learning outcomes. Instead, graduate work is distinguished from undergraduate work 
because it requires students to make use of the primary literature of the discipline to draw inferences 
for scholarly activity, whereas students in undergraduate programs might rely on the secondary 
literature, which presents an expert interpretation of the primary literature. Description of professional 
scholarship, research, and other creative activity completed by graduate students implies that these 
students will make a novel contribution to the primary literature. Students in applied graduate 
programs engage in professional activities and apply principles from the primary literature to solve 
real world problems in the discipline (professional practice and training). 

The DQP Grid articulates SLOs for associate's, bachelor's, and master's degree programs in each 
of six skill categories [3]. The contextual language of these SLOs rather than the verb selected supports 
the progressively more advanced nature of learning in degrees. The SLOs articulated for associate 
level degrees describe the acquisition and use of existing knowledge and methods in the discipline. 
In contrast, the language in outcomes described for master's level degrees describes the use of primary 
sources (disciplinary literature) to complete tasks, generation of novel solutions, and use of disciplinary 
research methods to make new contributions to the field. Contextual content included in SLOs written 
for bachelor's and master's degree programs describe increasing levels of independent, self-directed 
learning and professional activities [20]. 

Evidence for the advanced nature of learning in graduate degrees might also be based on the 
disciplinary content and skill described in a learning outcome. Some skills and specialized knowledge 
are reserved for master's and doctoral level education. For example, access to disciplinary assessment 
tools (e.g., personality and IQ tests) and training for the skills required to administer and interpret 
test results may be limited to students at the graduate level. However, the nature of this specialized 
knowledge is likely to be unique to disciplines and not obvious to broader audiences. 

5. Conclusions 

The enduring value of large collections of verbs is their ability to suggest measurable language 
when faculty write student learning outcomes. However, the lack of consistency in how lists align verbs 
with levels of thinking reduces the value of these collections as frameworks that will help programs 
articulate SLOS to describe lower- and higher-order thinking skills and differentiate between the 
learning achieved upon completion of associate's, bachelor's, and master's degree programs. Slobin 
and Miller refer to conversations between Alice and Humpty Dumpty (in Lewis Carroll's Through the 
Looking Glass) to illustrate the vexing problem posed by multiple meanings [21,22]. In this conversation, 
Humpty Dumpty asserts that he can make a word mean whatever he wants it to mean. Writers of 
SLOs might similarly claim that a verb means whatever level of skill they intend to describe. However, 
Miller counters that members of a community cannot enjoy the liberty claimed by Humpty Dumpty. 
If people in a community don't use words in the same way, they won't be able to understand each 
other [22] (p. 98). Thus, Bloom's taxonomy is a useful, but incomplete, framework for articulating 
measurable SLOs for advanced degrees. Instead, these verb collections function like a thesaurus or 
a rhyming dictionary. Authors use these lists to discover alternative words when they search for 
variety for their writing. But an author who uses a thesaurus has no guarantee that she will produce 
crisp prose or inspired poetry. Similarly, authors who use a list of Bloom's taxonomy verbs to write 
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measurable SLOs have no guarantee that the verbs they select for their learning outcomes will provide 
all the detail needed to clearly describe increasing levels of expertise. Authors must also consider the 
contextual content of the SLO. They should describe expectations for self-directed learning, immersion 
in the primary literature, engagement in professional practice, and the contributions to new knowledge 
that characterize expertise associated with an advanced degree. 
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Appendix A 

The web sites hosting collections of verbs for Bloom's taxonomy used in the analysis. 

https: / / www.cte.cornell.edu / documents / Assessment%20-%20Blooms%20Taxonomy%20Action% 
20Verbs.pdf 

http://chiron.valdosta.edu/whuitt/col/cogsys/bloom.html 
http: / / www.apu. edu / live_data / files /333 /blooms_taxonomy_action_verbs .pdf 
http: / / www.teach-nology.com / worksheets / time_savers/bloom / 
http://www.au.af.mil/au/awc/awcgate/edref/bloom.htm 

http:/ / teaching.uncc.edu/learning-resources/articles-books/best-practice/goals-objectives/ 
writing- obj ectives 

http: / / www.pamlicocc.edu / files/Bloom%20Taxonomy.pdf 

http: / / www.d.umn.edu / vcaa/assessment / documents/Bloomsverblist.doc 

http://www.marshall.edu/assessment/Resources/BloomsTaxonomyVerbs.pdf 

https: / /www.cloud.edu / Assets/PDFs / assessment / revised-blooms-chart.pdf 

http: / / www.uni.edu/adp / documents/bloomverbscognitiveaffectivepsychomotor.pdf 

http: / / www.worwic.edu/Media/Documents/Assessment/Bloom\Tl\textquoterights% 

20Taxonomy%20Breakdown.pdf 

http: / / www.personal.psu.edu/bxbl 1 / Objectives/ActionVerbsforObjectives.pdf 

http://www.cme.hsc.usf.edu/latestdocs/06-BloomsTaxonomy.pdf 

http: / / www.highperformancetrainingworks.com/EPSS/TechNotes/Bloom.htm 

http: / / www.farmington.kl2.mi.us/lon/pdf / resources/show_tell / show_tell_formB.pdf 

http://www.celt.iastate.edu/teaching/effective-teaching-practices/revised-blooms-taxonomy 

http://unex.uci.edu/pdfs/instructor/blooms_taxonomy.pdf 

http://www.schoolcraft.edu/pdfs/oca/blooms_taxonomy_verb_list.pdf 

http: / / www.aia. org / education / providers / AIAS076139 

http://www.trenton.kl2.nj.us/Downloads/blooms%20taxonomy%20verb%20wheel.pdf 
http: / / www.et.kent.edu/fpdc-db/files/05-verbs-for-bloom-s-taxonomy.pdf 
http://www.colorado.edu/sei/documents/Workshops/Handouts/Blooms_Taxonomy.pdf 
https: / / www.cmu.edu/teaching/resources/Teaching/CourseDesign/Objectives/ 
BloomsTaxonomyVerbs.pdf 

https://tulane.edu/som/ome/upload/blooms_tax_verbs.pdf 

http://cce.eng.ua.edu/files/2012/06/Bloom%C2%B9s-Taxonomy-Action-Verbs.pdf 

http: / / ww2.odu.edu / educ / roverbau/Bloom/blooms_taxonomy.htm 

https: / / www.qualitymatters.org/apphandoutverbwheell 112pdf / download / 

APPHandoutVerbWheellll2.pdf 

http: / / www.moreheadstate.edu/uploadedFiles / Sites/Main_Sites / Academics / COE/Educational_ 
Services_Unit/Blooms%20Verbs%20and%20Assessment%20Types.pdf 

https: / /www.dkit.ie / system/files/Bloom%27_s_Taxonomy_Level_and_Appropriate_Verbs.pdf 
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